[Torch FX] Compress PT2E Support #3663

anzr299 · 2025-09-22T14:43:32Z

Changes

Introduced a new API to offer weights compression algorithm for quantizers defined in torch.ao.
Currently only supports OpenVINO Quantizer.

Reason for changes

To support Quantizers defined in torch ao.

Related tickets

169342

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

…match in signatures in prepare_pt2e.

…_algo

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py

src/nncf/experimental/torch/fx/quantization/quantizer/__init__.py

tests/executorch/__init__.py

tests/executorch/test_quantizer.py

tests/executorch/observers.py

daniil-lyakhov

Can I see the PR with OpenVINOQuantizer?

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py

daniil-lyakhov · 2025-09-23T15:29:53Z

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py

+    ) -> torch.fx.GraphModule:
+        self._quantizer = quantizer


typehints an docstring are missing

src/nncf/quantization/algorithms/weight_compression/algorithm.py

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py

src/nncf/experimental/torch/fx/quantization/quantizer/openvino_adapter.py

Co-authored-by: Daniil Lyakhov <[email protected]>

nikita-savelyevv

Huge work, thanks @anzr299!

Mostly minor comments from my side. Overall the updated approach in src/nncf/quantization/algorithms/weight_compression/algorithm.py looks good in my opinion and does not change the logic of the algorithm.

The only significant difference I noticed is that ratio_defining_params are initialized with primary_config from the start, and then some of the parameters are converted back to backup precision after mixed precision algorithm. Before, it was the other way around. It looks a bit cumbersome during mixed precision assignment, but allows to avoid passing group_size_values which is an improvement compared to the previous approach.

src/nncf/quantization/algorithms/weight_compression/algorithm.py

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

tests/executorch/test_quantizer_compression.py

nikita-savelyevv · 2025-11-06T16:21:58Z

tests/executorch/test_quantizer_compression.py

+    quantizer_builder: Callable[..., OpenVINOQuantizer],
+    model_case: ModelCase,
+    quantizer_params,
+    pt2e_params,


I see that in this and some other cases below, pt2e_params argument is not used. Is this on purpose? Won't this result in unnecessary duplication of tests?

I initially left it there since all tests had a common fixture with TEST_MODELS. I have modified it now though, to use a different list to get arguments for test cases where pt2e_params is not required.

Please correct me if I'm wrong, but with TEST_MODELS_NO_PT2E defined as [(m, qparams) for m, qparams, _ in TEST_MODELS], it will still contain repeating entities

Oh yes you're right 🤦
it wasnt visible with current cases since there is only 1 element in the pt2e list.
Done

tests/torch/test_models/llama.py

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

Co-authored-by: Nikita Savelyev <[email protected]>

…e apply_parameters method

src/nncf/quantization/algorithms/weight_compression/algorithm.py

ljaljushkin

Awesome feature!

ljaljushkin · 2025-11-14T13:20:19Z

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py

+            model, graph, statistic_points, dataset, ratio_defining_params, all_weight_params
+        )
+        # Apply Mixed precision algorithm to ratio defining parameters
+        self._algo._apply_mixed_precision(ratio_defining_params, model, graph, statistic_points)


really minor and not blocking. I'd remove underscore in the name, since methods are used externally.

Ah yes great point

anzr299 added 3 commits September 22, 2025 17:22

init

190f9d5

fixes

c52fcca

add message for unsupported external quantizers

4e56cb5

anzr299 requested a review from a team as a code owner September 22, 2025 14:43

github-actions bot added the API Public API-impacting changes label Sep 22, 2025

anzr299 marked this pull request as draft September 22, 2025 14:56

daniil-lyakhov self-requested a review September 22, 2025 15:03

daniil-lyakhov reviewed Sep 22, 2025

View reviewed changes

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py Outdated Show resolved Hide resolved

anzr299 added 19 commits September 22, 2025 19:27

add algorithm

9651ceb

impotr openvino quantizer from nncf instead of executorch

14daeb5

Add observers and openvino quantizer to nncf

3746815

fix

0815dc5

minor fix

1b8d940

fix

7d35374

fix some more bugs; observers was importing from torchao. causing mis…

427ebc2

…match in signatures in prepare_pt2e.

add compress pt2e to init

24dbfb6

fix quantizer init file. Remove extra code.

4bb8c1a

small fix for the big problem:)

8902842

fix quantizer preset definition

3842538

fix openvino quantizer for ptq. call _algo instead of legacy _min_max…

2e70c2e

…_algo

fix quantizer defaults

b1c9aad

microfix

33fe01c

precommit fix

d8e1006

revert openvino quantizer to old

88a8472

create ovquantizer in executorch dir

7a8e51a

update executorch quantizer location.

fed5052

check if openvino quantizer has weight compression in openvino adapter

2866473

daniil-lyakhov requested changes Sep 23, 2025

View reviewed changes

daniil-lyakhov reviewed Sep 23, 2025

View reviewed changes

src/nncf/experimental/quantization/algorithms/weight_compression/algorithm.py Outdated Show resolved Hide resolved

anzr299 added 3 commits October 16, 2025 12:43

conftest precommit

6e379c8

update ref location for executorch

e45f796

define ratio in compress_pt2e API and not Quantizer itself; Update test

f2ece8c

daniil-lyakhov reviewed Oct 16, 2025

View reviewed changes

src/nncf/experimental/torch/fx/quantization/quantizer/openvino_adapter.py Outdated Show resolved Hide resolved

Apply suggestion from @daniil-lyakhov

387d69c

Co-authored-by: Daniil Lyakhov <[email protected]>

daniil-lyakhov self-requested a review November 5, 2025 12:15

anzr299 added 2 commits November 5, 2025 18:07

Merge branch 'openvinotoolkit:develop' into an/fx/compress_pt2e

4ace0df

pre-commit fix

00c8897

nikita-savelyevv reviewed Nov 6, 2025

View reviewed changes

daniil-lyakhov requested a review from ljaljushkin November 12, 2025 10:13

ljaljushkin reviewed Nov 12, 2025

View reviewed changes

tests/torch/test_models/llama.py Show resolved Hide resolved

ljaljushkin reviewed Nov 12, 2025

View reviewed changes

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py Show resolved Hide resolved

anzr299 and others added 5 commits November 13, 2025 15:57

Apply suggestions from code review

dd34b9b

Co-authored-by: Nikita Savelyev <[email protected]>

review changes

b9509bc

add mypy; review changes

d960d9a

precommit fix; seperate mixed precision algorithm application from th…

758bd67

…e apply_parameters method

add credit for transformers

193c404

daniil-lyakhov reviewed Nov 13, 2025

View reviewed changes

src/nncf/quantization/algorithms/weight_compression/algorithm.py Outdated Show resolved Hide resolved

daniil-lyakhov requested review from AlexanderDokuchaev and daniil-lyakhov November 13, 2025 14:43

anzr299 added 9 commits November 13, 2025 21:15

all optional arguemnts are keyword-only

8807c10

review changes

d86a90a

review changes

f2f01f2

executorch fix

0dc7f64

remove extra comments

f6d3739

fix duplication of tests

7ffa572

avoid square complexity

64803c3

review changes

9d25bed

change private methods to public which are used externally

859328e

ljaljushkin approved these changes Nov 14, 2025

View reviewed changes

[Torch FX] Compress PT2E Support #3663

Are you sure you want to change the base?

[Torch FX] Compress PT2E Support #3663

Conversation

anzr299 commented Sep 22, 2025

Changes

Reason for changes

Related tickets

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daniil-lyakhov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

daniil-lyakhov Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nikita-savelyevv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nikita-savelyevv Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

anzr299 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

nikita-savelyevv Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anzr299 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ljaljushkin left a comment

Choose a reason for hiding this comment

Uh oh!

ljaljushkin Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

anzr299 Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anzr299 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nikita-savelyevv Nov 14, 2025 •

edited

Loading

anzr299 Nov 14, 2025 •

edited

Loading